Assessing teachers' intervention-related expectations and self-efficacy: An examination of the factor structure of the implementation beliefs assessment.

Individual-level determinants are hypothesized to enable or prevent successful implementation of evidence-based practices, yet there are limited options for measuring theory-informed, individual-level determinants that influence teachers' and other implementers' delivery of school-based interventions. The goal of this study was to develop a self-report scale that measures variables that have been associated with initial and sustained behavior change related to school-based intervention implementation according to the health action process approach (HAPA). Participants were a nationally representative sample of kindergarten through Grade 12 public school teachers, stratified by grade level and geographical region. Item generation was based on a systematic review of the literature on outcome expectations and self-efficacy, the core constructs related to initiating and sustaining behavior change from the HAPA and in consultation with the theory developer. The sample was randomly split; half of the sample was used for exploratory factor analysis (EFA), and the other half was used for confirmatory factor analysis (CFA). The EFA resulted in a final factor structure of three dimensions of the Implementation Beliefs Assessment (IBA): (a) implementation self-efficacy, (b) positive outcome expectations, and (c) negative outcome expectations. This structure was supported in the other half of the sample using CFA. Additional analyses supported the reliability of IBA data. The IBA represents a step forward toward psychometrically sound measurement of factors associated with initial and sustained behavior change. Implications for future research are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Problem solving team interventions: Web-based and consultant-based experimental evaluations.

This research was designed to develop, implement, and evaluate an assessment and intervention protocol to increase problem-solving teams' (PSTs) adoption and implementation of evidence-based practices aimed at students with disruptive behavior problems. Participants included 15 PSTs. Adopting single-case design methodology, we examined whether a customized set of assessment and intervention consultant-led intervention procedures could be used to improve the activities, process, and recommendations of PSTs compared to a web-based intervention. We were interested in evaluating two variations of the problem-solving model based on the team initiated problem-solving (TIPS) approach. TIPS includes steps to successful problem solving and solution implementation for student academic and behavioral concerns. Based on visual analysis and statistical randomization tests, we found that a teleconsultation web-based model of PST intervention was not effective in improving the functioning of the PST. In contrast, a customized, consultation-led intervention model with PST facilitators that followed this approach was found to be effective in improving both the foundation and thoroughness of the PST's problem solving. Implications of future PST improvement models for practice and research are discussed.

Single-case intervention research design standards: Additional proposed upgrades and future directions.

Single-case intervention research design standards have evolved considerably over the past decade. These standards serve the dual role of assisting in single-case design (SCD) intervention research methodology and as guidelines for literature syntheses within a particular research domain. In a recent article (Kratochwill et al., 2021), we argued for a need to clarify key features of these standards. In this article we offer additional recommendations for SCD research and synthesis standards that have been either underdeveloped or missing in the conduct of research and in literature syntheses. Our recommendations are organized into three categories: expanding design standards, expanding evidence standards, and expanding the applications and consistency of SCDs. The recommendations we advance are for consideration for future standards, research design training, and they are especially important to guide the reporting of SCD intervention investigations as they enter the literature-synthesis phase of evidence-based practice initiatives.

Examining and Enhancing the Methodological Quality of Nonconcurrent Multiple-Baseline Designs.

In this article we respond to the recent recommendation of Slocum et al. (2022), who provided conceptual and methodological recommendations for reconsidering the credibility and validity of the nonconcurrent multiple-baseline design. We build on these recommendations and offer replication and randomization upgrades that should further improve the status of the nonconcurrent version of the design in standards and single-case design research. Although we suggest that the nonconcurrent version should be an acceptable methodological option for single-case design researchers, the traditional concurrent multiple-baseline design should generally be the design of choice.

Academic and Behavior Combined Support: Evaluation of an integrated supplemental intervention for early elementary students.

In current multi-tier models, students who are not responsive to universal instruction receive supplementary support. Despite most students having co-occurring academic and behavior challenges, their needs are often addressed through separate academic or behavior interventions. This approach may not only be costly for schools, but it also fails to acknowledge the well-documented link between behavior and academic performance. The purpose of this study was to evaluate Academic and Behavior Combined Support (ABC Support), a newly developed supplemental intervention that merges a combined focus on reading fluency and engagement. Six teachers implemented the intervention for 6-8 weeks with Grade 2 students. Students' oral fluency on standard and training reading passages and occurrence of engagement and disruptive behaviors during classroom reading instruction were measured repeatedly across baseline and intervention phases in a multiple-baseline design. Visual and statistical analyses revealed significant improvement from baseline to intervention for both reading and behavior outcomes. Post-intervention surveys and interviews revealed a high level of acceptability among teachers and students. Theoretical and empirical contributions as well as practice implications are addressed.

Single-case design standards: An update and proposed upgrades.

In this paper, we provide a critique focused on the What Works Clearinghouse (WWC) Standards for Single-Case Research Design (Standards 4.1). Specifically, we (a) recommend the use of visual-analysis to verify a single-case intervention study's design standards and to examine the study's operational issues, (b) identify limitations of the design-comparable effect-size measure and discuss related statistical matters, (c) review the applicability and practicality of Standards 4.1 to single-case designs (SCDs), and (d) recommend inclusion of content pertaining to diversity, equity, and inclusion in future standards. Within the historical context of the WWC Pilot Standards for Single-Case Design (1.0), we suggest that Standards 4.1 may best serve as standards for meta-analyses of SCDs but will need to make clear distinctions among the various types of SCD studies that are included in any research synthesis. In this regard, we argue for transparency in SCD studies that meet design standards and those that do not meet design standards in any meta-analysis emanating from the WWC. The intent of these recommendations is to advance the science of SCD research both in research synthesis and in promoting evidence-based practices.

Randomized Single-Case Intervention Designs and Analyses for Health Sciences Researchers: A Versatile Clinical Trials Companion.

AIM: An expository note introduces health sciences researchers to randomized single-case intervention designs, an adaptation of interrupted time-series methodology, and the staple of a scientifically credible small-sample research paradigm. METHODS: Detailed examples illustrating two different randomized single-case procedures are presented to highlight the techniques' advantages relative to small-sample nonparametric procedures that are commonly applied in the medical and health sciences fields. RESULTS: Numerous positive outcomes, based on both statistical simulation studies and actual intervention research investigations, support the applicability and value of these procedures. CONCLUSION: Randomized single-case intervention designs are recommended for consideration by health sciences researchers.

Incorporating feasibility protocols in intervention research.

In this paper, we review some dimensions of feasibility research. Feasibility research focuses on the intervention process and addresses questions about whether and how an intervention can be evaluated and implemented. Feasibility studies are implemented prior to conducting an outcome-focused pilot study or full-scale evaluation to test the effectiveness of an intervention. We propose a feasibility framework that includes 10 possible dimensions to evaluate in a feasibility trial, including (a) recruitment capability, (b) data collection procedures, (c) design procedures, (d) social validity, (e) practicality, (f) integration into existing systems, (g) adaptability, (h) implementation, (i) effectiveness, and (j) generalizability. Among these dimensions we offer some priorities that researchers can consider in establishing feasibility. Although feasibility investigations can advance evidence-based practice in psychology and education, we review current challenges for researchers to consider when incorporating a feasibility protocol into their intervention research agenda.

The efficacy of problem-solving consultation for homeschooled students with behavior concerns.

The purpose of this study was to explore the efficacy and acceptability of problem-solving consultation for homeschooling families with children who exhibited externalizing behavior problems. Three families participated, with multiple siblings participating from each family. Six children were male and 1 child was female. Children's ages ranged from 5 to 9 years old. Single-case experimental multiple baseline designs were used to evaluate the functional relation between implementation of behavior support plans within problem-solving consultation and children's externalizing behaviors. Direct observation data showed decreases in externalizing behaviors after the consultation and intervention process for 2 of the 3 families. The parents of the homeschooling children reported that the behavior support plans and consultation process were acceptable. Implications for future research and practice are presented. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Randomization procedures in single-case intervention research contexts: (Some of) "the rest of the story".

Following up on articles recently published in this journal, the present contribution tells (some of) "the rest of the story" about the value of randomization in single-case intervention research investigations. Invoking principles of internal, statistical-conclusion, and external validity, we begin by emphasizing the critical distinction between design randomization and analysis randomization, along with the necessary correspondence between the two. Four different types of single-case design-and-analysis randomization are then discussed. The persistent negative influence of serially dependent single-case outcome observations is highlighted, accompanied by examples of inappropriate applications of parametric and nonparametric tests that have appeared in the literature. We conclude by presenting valid applications of single-case randomization procedures in various single-case intervention contexts, with specific reference to a freely available Excel-based software package that can be accessed to incorporate the present randomization schemes into a wide variety of single-case intervention designs and analyses.

Evaluating implementation supports delivered within problem-solving consultation.

Teachers participating in problem-solving consultation often struggle to maintain adequate treatment fidelity, which is necessary to improve student outcomes. Low levels of treatment fidelity may result from implementation barriers, such as intervention compatibility, implementer skill, and implementer motivation. This study involves the evaluation of five implementation supports designed to address implementation barriers (i.e., Implementation Planning, Role Play, Participant Modeling, Raising Awareness, Motivational Consulting) within problem-solving consultation. Across 14 randomized individual single-case AB intervention designs, we evaluated the impact of these implementation supports on teacher treatment fidelity of classroom management plans and class-wide academic engagement and disruptive behavior. Visual analysis, descriptive statistics, and randomization test analyses suggest that these implementation supports have the potential to be broadly effective in improving teachers' fidelity and student outcomes. Teachers required a different number of supports to increase fidelity levels and rated the implementation supports positively. Implications of the study's findings are described.

Teleconsultation: The use of technology to improve evidence-based practices in rural communities.

Problem-solving consultation in schools has been found to be an effective method of service delivery to support teachers who are struggling to address student social-emotional behavioral (SEB) concerns. Despite its benefits, a number of barriers (e.g., lack of time and limited access to trained professionals) restrict the use of consultation within schools, especially in rural settings. The purpose of the study was to examine the impact of teleconsultation designed to improve behavior support to students living in rural communities. Both student outcomes and teacher perceptions were evaluated. Results indicated that (a) student disruptive behaviors improved through the implementation of an individualized behavior support plan developed through teleconsultation, and (b) teachers found the teleconsultation experience acceptable and feasible. As the demands placed on psychologists and the quality of videoconferencing continue to increase, teleconsultation is becoming a viable option for service delivery within rural school settings.

A survey of publication practices of single-case design researchers when treatments have small or large effects.

The published literature often underrepresents studies that do not find evidence for a treatment effect; this is often called publication bias. Literature reviews that fail to include such studies may overestimate the size of an effect. Only a few studies have examined publication bias in single-case design (SCD) research, but those studies suggest that publication bias may occur. This study surveyed SCD researchers about publication preferences in response to simulated SCD results that show a range of small to large effects. Results suggest that SCD researchers are more likely to submit manuscripts that show large effects for publication and are more likely to recommend acceptance of manuscripts that show large effects when they act as a reviewer. A nontrivial minority of SCD researchers (4% to 15%) would drop 1 or 2 cases from the study if the effect size is small and then submit for publication. This article ends with a discussion of implications for publication practices in SCD research.

An exploratory investigation of teachers' intervention planning and perceived implementation barriers.

Increasingly teachers are the primary implementer responsible for providing evidence-based interventions to students. However, there is little knowledge regarding the extent to which teachers plan for intervention implementation, receive implementation support, or identify and address implementation barriers. This study explores survey data from over 1200 preschool through grade 12 teachers from 46 public school districts in a Northeastern state. Results indicate that teachers spend significant time engaging in intervention-related behavior and may be a primary source responsible for selecting student interventions. However, the current extent to which they plan for implementation and present levels of implementation support are inadequate to produce high levels of sustained intervention implementation. In addition, almost 60% of implementation barriers reported related to aspects of the intervention itself. Findings from this study provide guidance for future research and preliminary recommendations for ameliorating implementation barriers and proactively supporting treatment integrity in schools.

Increasing teacher treatment integrity of behavior support plans through consultation and implementation planning.

School psychologists commonly provide intervention services to children through consultation with teachers. Data suggest, however, that many teacher consultees struggle to implement these interventions. There are relatively few evidence-based strategies for promoting teacher consultees' intervention implementation. The purpose of this study was to evaluate Implementation Planning as a strategy for increasing the adherence and quality with which teacher consultees implemented behavior support plans. Implementation Planning involves adapting intervention steps to the implementation context, providing detailed logistical planning, as well as identifying implementation barriers and developing strategies to address them. Results indicated that teachers' implementation adherence levels increased and quality levels increased with corresponding decreases in variability, compared to standard behavioral consultation. Implications for future research on treatment integrity are discussed.

Size and consistency of problem-solving consultation outcomes: an empirical analysis.

In this study, we analyzed extant data to evaluate the variability and magnitude of students' behavior change outcomes (academic, social, and behavioral) produced by consultants through problem-solving consultation with teachers. Research questions were twofold: (a) Do consultants produce consistent and sizeable positive student outcomes across their cases as measured through direct and frequent assessment? and (b) What proportion of variability in student outcomes is attributable to consultants? Analyses of extant data collected from problem-solving consultation outcome studies that used single-case, time-series AB designs with multiple participants were analyzed. Four such studies ultimately met the inclusion criteria for the extant data, comprising 124 consultants who worked with 302 school teachers regarding 453 individual students. Consultants constituted the independent variable, while the primary dependent variable was a descriptive effect size based on student behavior change as measured by (a) curriculum-based measures, (b) permanent products, or (c) direct observations. Primary analyses involved visual and statistical evaluation of effect size magnitude and variability observed within and between consultants and studies. Given the nested nature of the data, multilevel analyses were used to assess consultant effects on student outcomes. Results suggest that consultants consistently produced positive effect sizes on average across their cases, but outcomes varied between consultants. Findings also indicated that consultants, teachers, and the corresponding studies accounted for a significant proportion of variability in student outcomes. This investigation advances the use of multilevel and integrative data analyses to evaluate consultation outcomes and extends research on problem-solving consultation, consultant effects, and meta-analysis of case study AB designs. Practical implications for evaluating consultation service delivery in school settings are also discussed.

Meta- and statistical analysis of single-case intervention research data: quantitative gifts and a wish list.

In this commentary, we add to the spirit of the articles appearing in the special series devoted to meta- and statistical analysis of single-case intervention-design data. Following a brief discussion of historical factors leading to our initial involvement in statistical analysis of such data, we discuss: (a) the value added by including statistical-analysis recommendations in the What Works Clearinghouse Standards for single-case intervention designs; (b) the importance of visual analysis in single-case intervention research, along with the distinctive role that could be played by single-case effect-size measures; and (c) the elevated internal validity and statistical-conclusion validity afforded by the incorporation of various forms of randomization into basic single-case design structures. For the future, we envision more widespread application of quantitative analyses, as critical adjuncts to visual analysis, in both primary single-case intervention research studies and literature reviews in the behavioral, educational, and health sciences.

Applying adult behavior change theory to support mediator-based intervention implementation.

A majority of evidence-based interventions in schools are delivered through consultation models and are implemented by a mediator, such as a teacher. Research indicates that mediators do not always adequately implement adopted evidence-based interventions, limiting their effectiveness in transforming student outcomes. We propose that to transform student outcomes through evidence-based practice, conceptualization of mediators' intervention implementation must move beyond quantification of discrete intervention steps implemented. Intervention implementation requires behavior change and thus can be conceptualized as an adult behavior change process. The purpose of this article is to illustrate how adult behavior change theory may inform how intervention implementation is conceptualized, facilitated, and supported. An empirically supported theory of adult behavior change from health psychology, the Health Action Process Approach, and how it has informed development of PRIME (Planning Realistic Intervention Implementation and Maintenance by Educators), a system of supports to facilitate mediators' implementation of school-based interventions, are introduced. A case study demonstrating the application of PRIME with a public school teacher implementing a behavior support plan is presented. Implications for future research are discussed.

Nonparametric statistical tests for single-case systematic and randomized ABAB AB and alternating treatment intervention designs: new developments, new directions.

In this four-investigation Monte Carlo simulation study, we examined the properties of nonparametric randomization and permutation statistical tests applied to single-case ABAB…AB and alternating treatment designs based on either systematically alternating or randomly determined phase assignments. Contrary to previous admonitions, when appropriately conceptualized nonparametric tests were conducted the systematic designs maintained Type I error rates at or less than their nominal values whenever either no or positive autocorrelation was present in the series. Statistical power for all designs varied as a function of both the number of phases and the number of observations per phase. It was concluded that to increase methodological rigor and statistical precision in the face of autocorrelation, (a) whenever possible, researchers should adopt a randomization scheme in assigning the orders in which the A and B phases are administered and (b) in situations where randomized phase assignment is not feasible, researchers should adopt a systematic phase assignment scheme consisting of individually alternating A and B observations, with a cost of sacrificing some degree of statistical power. Implications for single-case intervention research in school psychology are discussed.

Comments on "Distinguishing science from pseudoscience in school psychology:" Evidence-based interventions for grandiose bragging.

The purpose of this article is to provide some perspectives on Lilienfeld, Ammirati, and David's (2012) paper on distinguishing science from pseudoscience in school psychology. In many respects their work represents an intervention for "grandiose bragging," a problem that has occasionally occurred when various non-evidence-based or discredited interventions receive sensationalized positive endorsement for adoption in school psychology practice. In this paper, the implications of the Lilienfeld et al. work are discussed within the context of the scientist-practitioner gap, scientific thinking and evaluation of scientific thinking, and negative results research. The authors have advanced our thinking on evidence-based practices in school psychology and education.